Mondrian Motion

In this paper we develop a framework for motion estimation and stereo matching that allows explicit reasoning about occlusion relationships between untextured surface patches. We begin by studying a simplified problem, namely a Mondrian World that consists of uniformly colored rectangles translating with constant velocities at different depths. We present methods for inferring the extent, velocity, and relative depth of each region from a small number of frames, even in the presence of significant occlusion. We then show how the underlying assumptions can be relaxed so that our methods can be used to infer occlusion relationships and depth orderings among untextured surfaces in real-world motion and stereo image sequences.